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METHOD OF EVALUATING COSMETIC PRODUCTS ON A 
CONSUMER WITH FUTURE PREDICTIVE TRANSFORMATION 



Field of the Invention 

[0001] The invention concerns a method for transforming an image of a 

consumer's skin, especially facial skin, to an image with a consumer- 
defined composite cosmetic attribute. 

The Related Art 

[0002] Cosmetic companies have developed on-counter visualization tools to 



show wrinkles and how they can worsen, based on increasing the 
length and width of the wrinkles. These tools are not based on actual 
consumer or clinical data, or actual studies of the progression of a 
wrinkle over time. In addition, many companies have the technology 
to display actual images with color changes to the face, without 
showing how the face would appear as a result of improving a 
consumer-defined composite cosmetic attribute. See for instance JP 
7100126 and JP 5196436 (Shiseido). Another patent from Shiseido 
(JP 63006461) provides recommendations from a video editing 
system based on the roughness of a replica taken on counter. 




BACKGROUND OF THE INVENTION 
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JP 8299288 (Kanebo) describes "spot" measurements, which may be 
related to dryness, oiliness, or color over a small (mm size) region on 
the skin. While these measurements are on-counter diagnostics, they 
do not include an overall attribute visualization of the face as in the 
present invention. 

Clinique has a web site for product recommendation, based on user 
input. 'It does not involve actual changes or expert ratings in any 
composite cosmetic attribute or healthy skin. Revlon in U.S. Patent 
5,163,010 describes a similar system for recommending custom 
mixed products based on surveys at the point of sale. 

Studies exist on the relationship of facial features to the perception of 
age and gender. See for instance Hawkins SS, Perrett DI, Burt DM, 
Rowland DA and Murahata RI. "Prototypes of facial attributes 
developed through image averaging techniques", Intern. J. of Cos. 
Science, 21; 159-166 (1991); Rowland DA, Perrett DI, Burt DM, Lee 
KJ and Akamatsu S. "Transforming Facial Images in 2 and 3-D", 
Imagina 97-Conferences-Actes/Proceedings, Pages 1-8, February 
1997; Burt DM, Perrett DI. "Perceptual asymmetries in judgments of 
facial attractiveness, age, gender, 'speech and expression" 
Neuropsychologia 35(5):685-693, 1997; Burt DM, Perrett DI. 
"Perception of age in adult Caucasian male faces: computer graphic 
manipulation of shape and colour information." Proc. R. Soc. Lond. B 
259:137-143, 1995; Rowland DA, Perrett DI. "Manipulating facial 
appearance through shape and color." IEEE Computer Graphics and 
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Applications 15(5):70-76, 1995; Perrett DI f May KA, Yoshikawa S. 
"Facial shape and judgments of female attractiveness." Nature 
368:239-242, 1994. These articles have advanced Imaging 
technology but have not been applied specifically to model individual 
attributes, which taken collectively define a composite cosmetic 
attribute (e.g., "healthy" skin), sought by a consumer. 



[0006] Accordingly, it is an object of the present invention to provide a 

method for presenting an image of a consumer's skin, especially that 
of the face, in a manner reflecting a cosmetic product's effect over a 
period of time. 

[0007] Another object of the present invention is to provide a method for 
evaluating a cosmetic product on a consumer's face or other body 
parts through image transformation over a period of time. 

[0008] Still another object of the present invention is to provide a method 
wherein an image of an individual's actual face is transformed over 
time and the effects of a cosmetic product on color, sags, wrinkles, 
texture and radiance can be demonstrated. 



[0009] Yet another object of the present invention is to provide a method for 
demonstrating to a potential customer the benefit of certain cosmetic 
products when used over a period of time and allowing comparison 
with the effects of non-treatment. 
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[00010] These and other objects of the present invention will become more 
fully apparent from consideration of the following summary and 
detailed discussion. 



SUMMARY OF THE INVENTION 



[00011] A method is provided for demonstrating effectiveness of a cosmetic 
product on a consumer's body parts when applied over a period of 
time, the method including: 
10 (i) selecting a cosmetic product for trial on the consumer's 

body and identifying a body feature being observed 
over the time; 

(ii) capturing an image of the body part; 

(iii) displaying the image on a monitor; 

15 (iv) digitally transforming the selected body feature on the 

displayed image in conformance with a predicted effect 
of the cosmetic product and displaying the digitally 
transformed image; 

(v) allowing the consumer to compare results of the 
20 transformed versus the displayed image; and 

(vi) optionally repeating steps (iv) and (v) till the consumer 
has chosen an optimal transformation. 



[00012] Among the body features which may be manipulated are those of 
25 texture (pores), sags/lines/wrinkles and radiance/color. In most 

instances the body part selected for transformation will be that of the 
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face or portions of the face. Comparison of images in step (v) may 
be tiled on the same screen. Alternatively the displayed and 
transformed images may be on separate screens. They may also be 
viewable on consecutive screens, usually by a single dick of a 
computer mouse or key. The tile option is preferred. This mode may 
also include having more than one transformed image placed on a 
screen with the original displayed image. For instance, three 
transformed images can be placed adjacent to a displayed image 
wherein each of the transformed images represents the effects of 
using a particular cosmetic product over one month, two months and 
one year. 

[00013] The present invention is also concerned with a method for identifying 
cosmetic needs of a consumer with respect to physical changes that 
are personal to the consumer, the method including: 

(i) gathering information from the consumer concerning 
the consumer's cosmetic needs in treating a body 
feature; 

(ii) capturing an image of the consumer's body feature; 

(iii) displaying the image on a monitor; 

(iv) digitally transforming in a manner reflecting the effect 
of a recommended cosmetic over a period of time on a 
property of the body feature selected from color, sags, 
wrinkles, texture, radiance and combinations thereof, 
and displaying the transformed image; 
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(v) requiring the consumer to select between transformed 
and displayed images; and 

(vi) optionally repeating steps (iv) and (v) till the consumer 
chooses an optimal transformation. 
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[00014] A further aspect of the invention is a kit for displaying a consumer's 
skin image on a computer screen and transforming the image into 
consumer-defined composite attributes. Included is a camera for 
obtaining a high resolution image of the consumer's skin or hair, and 
a programmable device for receiving the image, transforming the 
image to an optimum image by subtracting from or adding to the 
image a number of difference images constructed from prototypes for 
each attribute of a composite cosmetic attribute. 
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In a preferred embodiment, the programmable device also evaluates 
the direction and amount of the change for each attribute of the 
composite cosmetic attribute, so that a skin cosmetic product or 
regimen can be recommended to a consumer. Recommendation may 
also be done by a programmable device. 

When the method focuses upon aging attributes, improvement from 
one image to another can be expressed as a percentage 
improvement. According to the present invention, it is possible to 
replace the percentage concept simply with a value of number of 
years reduction in apparent appearance. 
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[00017] The invention employs a method for obtaining a transform of a 
composite cosmetic attribute, the method including: 

(a) identifying, preferably via a consumer study, individual 
attributes of a composite cosmetic attribute and grading 
a plurality of subjects on each attribute; 

(b) computing an average for subjects rated low and an 
average for subjects rated high to obtain the high 
prototype and the low prototype for each attribute; 

(c) computing a difference image for the individual attribute 
between a high and a low prototype; 

(d) obtaining an optimum image by subtracting from or 
adding to an actual captured image increments of the 
difference image. 

[00018] Preferably, an optimum image is obtained by simultaneously 
subtracting/adding difference images for several or all attributes that 
are identified. Thus the captured image is transformed by a consumer 
to an optimum image with a composite cosmetic attribute, to a degree 
desired by a consumer. The inventive device and method allow 
consumers to regulate the degree to which they adjust each 
component of a composite cosmetic attribute. ,In the most preferred 
embodiment of the invention, the method is consumer-driven: first, 
the composite cosmetic attribute is defined by a consumer (preferably, 
by a large consumer panel); then prototypes are obtained from 
consumer gradings (preferably, expert panel grading of a large 
number of consumers), and finally a consumer defines her/his image 
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with a composite cosmetic attribute, being able to adjust each 
individual component of a composite cosmetic attribute. 

[00019] In the preferred embodiment of the invention, the transformed image 
defined by a consumer is subsequently analyzed by a cosmetics 
manufacturer (or a sales representative) to evaluate the amount and 
direction of the change for each attribute of the composite cosmetic 
attribute. Subsequently, a cosmetics manufacturer (or sales 
representative thereof) can recommend a skin cosmetic product or a 
skin cosmetic regimen which would assist the consumer in attaining 
the transformed image. 



[00020] The inventive method and kit are capable pf displaying a transformed 
image which contains an attribute not at all present in the captured 
image. For instance, a consumer may see glow in the transformed 
image, even if there was no glow at all in the captured image. 

[00021] Yet a further aspect of the present invention involves a method for 
identifying progress in the affect of an applied cosmetic product upon 
a consumer which affect is personal to that consumer, the method 
including: 

(i) selecting a cosmetic product for trial on the consumer's 
body and identifying a body feature being observed 
over a period of time; 

(ii) capturing an image of the body feature; 

(iii) displaying the image on a monitor; 
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(iv) capturing a further image of the body feature at a later 
time after the product has been applied by the 
consumer; 

(v) allowing the consumer to compare results between the 
captured and later time captured images; and 

(vi) optionally digitally transforming the captured image of 
step (ii) in conformance with a predicted affect of the 
cosmetic product and displaying the digitally 
transformed image thereby allowing comparison to the 
later time captured image representing an actual 
cosmetic product treated body feature as obtained from 
step (iv). 

DETAILED DESCRIPTION OF THE INVENTION 

[00022] Now it has been found that a consumer's actual face or other body 
part can be digitally transformed through a computer program to 
demonstrate effects of a cosmetic product treatment. The treatment 
may simply be an application of different colors onto skin (such as 
evaluating the different shades of lipstick, blush* eye or lip liners, 
foundation and mascara) or even hair. The latter may include 
coloring, hair growth or even deputation. Transformations of the 
present invention take into account not only topography and shape of 
a face but also its color and texture. An important aspect of the 
invention is that transformed and originally displayed images are 
placed either beside one another or on adjoining screen views. The 
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consumer is forced to pick between two of the images. Forced choice 
ordinarily is between a transformed and an original displayed image 
but may also occur between different ones of the transformed 
images. 

[00023] According to the present invention a consumer is assisted in 

evaluating cosmetic products with a future predictive transformation 
obtainable in the following manner. High resolution images are 
captured of body sites which may benefit from the cosmetic products. 
Feature points are then semi-automatically delineated along the skin 
surface of those sites. Images have been grouped by one or more of 
a series of attribute scales to define an Attribute S. 

[00024] Based on consumer perception studies, it was determined that female 
consumers wanted healthy looking skin. "Healthy" skin is a 
composite cosmetic attribute, which has not been previously defined. 
The term "skin" as used herein includes skin on the face, neck, 
hands, chest, arms and legs. Attribute S was developed using a set 
of scales to define healthy skin. 

[00025] Qualitative language generation with Caucasian females in the US 
was performed to determine the individual attributes of healthy 
looking facial skin, and expert graders were trained based on the 
consumer language. 
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[00026] The study classified various cosmetic facial attributes as consumer- 
perceivable attributes that contribute to the overall appearance of 
cosmetic health. Typical attributes included lines and wrinkles, pores, 
blemishes, softness, silkiness and glow/color. Thus, the individual 
indicia of healthy facial skin were defined objectively and by a 
consumer. 

[00027] Three attributes (glow/color, lines and wrinkles, and pores) were then 
worked on as follows: Expert evaluations and photographs were 
taken of 80 panelists with no treatment/product use. The panelists 
all wore white hair caps, black bibs, and had cleansed their faces with 
their normal cleansing products 2 hours prior to their arrival at the 
study site. The photographs were cecorded to PhotoCD for 
processing. From these images, facial averages were computed for 
panelists with the 9 lowest ratings for healthy glow/color, and the 
highest ratings for lines and wrinkles, and pores. The methods for 
computing facial averages have been previously described in the 
literature. These facial averages, or prototypes, provided insight into 
the primary regions of interest for each of the three attributes. 
Difference images were than constructed by subtracting the low 
prototype from the high prototype. This' 1 procedure was repeated for 
19 attributes total, which together defined a composite cosmetic 
attribute "healthy skin." 
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[00028] Novice graders were prompted to "make faces look healthy/ using 
varying percentages of difference images to apply a positive or 
negative transform of a particular attribute. Subjects were shown 9 
faces in each of the three attribute categories (glow/color, pores, and 

5 lines and wrinkles). The transforms were shown in random order 

with counterbalanced directions of transforms and mid-range points. 
Three faces were rated high, three mid-range, and three low for each 
category by expert graders. It was confirmed that novice graders 
were able to describe what they were seeing in the process of 

10 transforming the image and that their description corresponded well 

to the attributes developed by expert graders. Known physiological 
processes were detected on individual attributes, e.g. aging for 
lines/wrinkles. 
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The novice graders recognized all of the transforms as changing 
perceived facial health in the transformed attribute along the correct 
transform direction. For example, subtraction of an attribute resulted 
in a decreased perception of healthy glow, pores, or lines and 
wrinkles. 



[00030] The same experiment performed with novice graders was repeated 
with panel experts. Expert graders were able to identify the attribute 
that was used in obtaining a transformed image. Like novice graders, 
expert graders were able to identify the correct direction for an 
25 attribute resulting in a transformed image. 
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[00031] According to the present invention, a consumer has their picture 
taken at a sales counter. By adjusting the captured image through 
varying amounts and directions of difference images, an operator 
obtains a consumer's definition of their own ideal healthy skin. 

5 Customized product information can then be given to the consumer 

at the counter to help achieve this ideal. Clinically proven changes in 
products could also be shown to the consumer on their own image 
using this point of purchase interactive imaging system. Instead of a 
retail counter, the whole process may be conducted from a 

10 consumer's home via Internet transmissions and by use of a 

consumer's own digital camera. 

It should be understood that the specific forms of the invention herein 
illustrated and described are intended to be representative only. 
Changes, including but not limited to those suggested in this 
specification, may be made in the illustrated embodiments without 
departing from the clear teachings of the disclosure. Accordingly, 
reference should be made to the following appended claims in 
determining the full scope of the invention. 
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